Abstract: This paper proposed the application solutions of coal mine safety surveillance and activity recognition system based on video. The paper analyzed the practical requirement of coal mine safety surveillance, realized the functional modules for object detection, feature extraction, activity classification and activity identification, surveillance information management. By using GMM technology, the paper extract the moving target in foreground, applied LBP feature for object description and classified the activity with SVM. Based on Spring MVC, the paper implemented a complete framework of management system providing a solution for coal mine safety surveillance management information to improve the efficiency of the surveillance management. This management system could be used for timely discovery of coal mine safety emergencis and reduction the probability of occurrence of coal mine safety accidents.